Class 22

Expected Counts

If we assume \(H_0\): Independence, how can we estimate the data expected under the null hypothesis?

Proportions

Calculating Expected Counts

Using proportions…

\[ \operatorname{Expected}_{\operatorname{A and B}}=P(A) \times P(B) \times n \]

The Test Statistic

The Chi-Square (\(\chi^2\)) test statistic is the sum of the squared difference between observed and expected value divided by the expected value for all combinations of categories.

\[ \chi^2_{df} =\sum^k_{i=1} \frac{{\left( \operatorname{observed} - \operatorname{expected} \right)}^2}{\operatorname{expected}} \]

Degrees of Freedom

For a two-way table, the degrees of freedom for a \(\chi^2\) test statistic are…

\[ \begin{aligned} df &= \left( n_{\operatorname{rows}} - 1 \right) \times \left( n_{\operatorname{cols}} - 1 \right) \\ &= \left( R-1 \right) \times \left( C-1 \right) \end{aligned} \]

…where \(R\) is the number of rows in the table and \(C\) is the number of columns.

P-Values in R

You can find p-values in R using the pchisq() function, which takes a test statistic (q) and the degrees of freedom (df) as parameters.

pchisq(2, df = 3, lower.tail = F)

[1] 0.5724067

For the \(\chi^2\) test, we always use the upper tail, or the probability of seeing a result more extreme than what we observed.

Conditions

The \(\chi^2\) test is most appropriate for large sample sizes.

• > 20% of expected counts are >5.

Limitations

• Neither the \(\chi^2\) test nor the Fisher’s Exact Test will tell you the nature of the relationship between your 2 categorical variables.

• Additional tests are needed to determine if the outcomes are dependent on variable 1, variable 2, or both.

Sample Size Estimation for Proportions

You can use the margin of error calculation to estimate the sample size needed to detect a given difference in proportions.

\[ \operatorname{margin of error} = Z^* \times \sqrt{\frac{p(1-p)}{n}} \]

Example: Sample Size

We want to know if people favor candidate 1 or candidate 2 (\(H_0\): \(P(C_1)=P(C_2)\)), but it will be a very close race. If we want to find 52% for candidate 1 vs 48% for candidate 2, what size sample do we need?

Example: Sample Size

We want to know if people favor candidate 1 or candidate 2 (\(H_0\): \(P(C_1)=P(C_2)\)), but it will be a very close race. If we want to find 52% for candidate 1 vs 48% for candidate 2, what size sample do we need?

Polling History

Polling Error

Modern Polling

Sampling

• Probabilitistic samples have less error than non-probabilistic samples

• Results coming out are only as good as the data going in

• Can you reliably, validly, generalizably describe the US when \(n=1000\)? \(n=2000\)?

Weighting

• Some demographics are hard to characterize in a pure random sample
• Some categories are oversampled while others are undersampled, not always intentionally
• Sampled data is adjusted to more closely match the study population characteristics

Likely Voters

• ~ 1/3 of eligible Americans do not vote in presidential elections
• Many people feel social pressure to say they’ll vote even if it’s unlikely
• The study population is not static over time!
• Popular vote does not necessarily win

Sources of Error

Other Sources of Error

• Interviewer

• Responder

• Survey

Aggregators

• Combining results from multiple surveys may improve accuracy

• Are all polls created equal?

• If the underlying assumptions are faulty, more data won’t improve the quality

Communicating Results

• Transparency

• Point estimates? Confidence intervals?

• Certainty